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Abstract 

We consider how an agent should update her uncertainty when it is represented by a 
set V of probabihty distributions and the agent observes that a random variable X takes on 
value X, given that the agent makes decisions using the minimax criterion, perhaps the best- 
studied and most commonly-used criterion in the literature. We adopt a game-theoretic 
framework, where the agent plays against a bookie, who chooses some distribution from 
V. We consider two reasonable games that differ in what the bookie knows when he 
makes his choice. Anomalies that have been observed before, like time inconsistency, can 
be understood as arising because different games are being played, against bookies with 
different information. We characterize the important special cases in which the optimal 
decision rules according to the minimax criterion amount to either conditioning or simply 
ignoring the information. Finally, we consider the relationship between conditioning and 
calibration when uncertainty is described by sets of probabilities. 



1 Introduction 

Suppose that an agent models her uncertainty about a domain using a set V of probability distri- 
butions. How should the agent make decisions? Perhaps the best-studied and most commonly- 
used approach in the literature is to use the minimax criterion r Wald 1950llGardenfors and Sahlin 1982t 
IGilboa and Schmeidler 19891 . According to the minimax criterion, action a\ is preferred to ac- 
tion 02 if the worst-case expected loss of a\ (with respect to all the probability distributions in 
the set V under consideration) is better than the worst-case expected loss of Thus, the action 
chosen is the one with the best worst-case outcome. 

We are often interested in making decisions, not just in a static situation, but in a more 
dynamic situation, where the agent may make some observations, or learn some informa- 
tion. This leads to an obvious question: If the agent represents her uncertainty using a set 
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V of probability distributions, how should she update V in light of observing that random 
variable X takes on value x? Perhaps the standard answer is to condition each distribution 
in P on X = x (more precisely, to condition those distributions in V that give X = x 
positive probability on X = x), and adopt the resulting set of conditional distributions V \ 
X = X d& her representation of uncertainty. As has been pointed out by several authors, this 
sometimes leads to a phenomenon called dilation JAugustin 2003 [ |Cozman and Walley 2001 



Herron, Seidenfeld, andWasserman 1997t ISeidenfeld and Wasserman 199311 : the agent may 



have substantial knowledge about some other random variable Y before observing X = x, 
but know significantly less after conditioning. Walley [119911 p. 299] gives a simple example 
of dilation: suppose that a fair coin is tossed twice, where the second toss may depend in an 
arbitrary way on the first. (In particular, the tosses might be guaranteed to be identical, or 
guaranteed to be different.) If X represents the outcome of the first toss and Y represents the 
outcome of the second toss, then before observing X, the agent believes that the probability 
that Y is heads is 1/2, while after observing X, the agent believes that the probability that Y 
is heads can be an arbitrary element of [0, 1]. 

While, as this example and others provided by Walley show, such dilation can be quite 
reasonable, it interacts rather badly with the minimax criterion, leading to anomalous behavior 
that has been called time inconsistency [[Grunwald and Halpem 2004] ISeidenfeld 200411 : the 



minimax-optimal conditional decision rule before the value of X is observed (which has the 
form "If X = then do ai; if X = 1 then do 02; . . . ") may be different from the minimax 
decision rule obtained after conditioning. For example, the minimax-optimal conditional deci- 
sion rule may say "If X = then do ai", but the minimax-optimal decision rule conditional 
on observing X = may be 02. (See Example 12. II ) If uncertainty is modeled using a single 
distribution, such time inconsistency cannot arise. 

To understand this phenomenon better, we model the decision problem as a game between 
the agent and a bookie. It turns out that there is more than one possible game that can be 
considered, depending on what information the bookie has. We focus on two (closely related) 
games here. In the first game, the bookie chooses a distribution from V before the agent moves. 
We show that the Nash equilibrium of this game leads to a minimax decision rule. (Indeed, 
this can be viewed as a justification of using the minimax criterion). However, in this game, 
conditioning on the information is not always optimalQ 

In the second game, the bookie gets to choose the distribution after the value of X is 
observed. Again, in this game, the Nash equilibrium leads to the use of minimax, but now 
conditioning is the right thing to do. 

If P is a singleton, the two games coincide (since there is only one choice the bookie can 
make, and the agent knows what it is). Not surprisingly, conditioning is the appropriate thing 
to do in this case. The moral of this analysis is that, when uncertainty is characterized by a 
set of distributions, if the agent is making decision using the minimax criterion, then the right 
decision depends on the game being played. The agent must consider if she is trying to protect 



'in some other senses of the words "conditioning" and "optimal," conditioning on the information is always 
optimal. This is discussed further in Section|6l 
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herself against an adversary who knows the value of X = x when choosing the distribution or 
one that does not know the value of X = x. 



In earlier work [Griinwald and Halpern 2004 1 (GH from now on), we essentially considered 
the first game, and showed that, in this game, conditioning was not always the right thing to do 
when using the minimax criterion. Indeed, we showed there are sets V and games for which 
the minimax-optimal decision rule is to simply ignore the information. Our analysis of the 
first game lets us go beyond GH here in two ways. First, we characterize exactly when it is 
minimax optimal to ignore information. Second, we provide a simple sufficient condition for 
when conditioning on the information is minimax optimal. 

Ignoring the information can be viewed as the result of conditioning; not conditioning 
on the information, but conditioning on the whole space. This leads to a natural question: 
suppose that when we observe x, we condition on the event that X E C{x), where C{x) is 
some set containing x, but not necessarily equal to {x}. Is this variant of conditioning, an 
approach we call C -conditioning, always minimax optimal in the first game? That is, is it 
always optimal to condition on something! As we show by considering the well-known Monty 
Hall Problem (Example 15. 31) . this is not the case in general. Nevertheless, C-conditioning has 
some interesting properties: it is closely related to the concept of calibration HDawid 1 9821. 
Calibration is usually defined in terms of empirical data. To explain what it means, consider 
an agent that is a weather forecaster on your local television station. Every night the forecaster 
makes a prediction about whether or not it will rain the next day in the area where you live. 
She does this by asserting that the probability of rain is p, where p E {0, 0.1, ... , 0.9, 1}. How 
should we interpret these probabilities? The usual interpretation is that, in the long run, on 
those days at which the weather forecaster predict probability p, it will rain approximately 
100p% of the time HDawid 1982L Thus, for example, among all days for which she predicted 
0.1, the fraction of days with rain was close to 0.1. A weather forecaster with this property is 
called calibrated. 

Up to now, calibration has been considered only when uncertainty is characterized by a 
single distribution. We generalize the notion of calibration to our setting, where uncertainty 
is characterized by a set of distributions. We then show that a rule for updating a set of prob- 
abilities is guaranteed to be calibrated if and only if it is an instance of C-conditioning. In 
combination with our earlier results, this implies that if calibration is considered essential, then 
an update rule may sometimes result in decisions that are not minimax optimal. 

The rest of this paper is organized as follows. In Section[2l we define the basic framework. 
In Section [3l we formally define the two games described above and show that the minimax- 
optimal decision rule gives a Nash equilibrium. In Section |4l we characterize the minimax- 
optimal decision rule for the first game, in which the bookie chooses a distribution before X is 
observed. In Section [5l we discuss C-conditioning and calibration. 
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2 Notation and Definitions 



Preliminaries: For ease of exposition, we assume throughout this paper that we are inter- 
ested in two random variables, X and Y, which can take values in spaces X and 3^, respectively. 

V always denotes a set of distributions on X x y; that is, V A(X x y), where, as usual, 
A{S) denotes the set of probability distributions on S. For ease of exposition, we assume that 

V is a closed set; this is a standard assumption in the literature that seems quite natural in our 
applications, and makes the statement of our results simpler. If Pr G A(X x y), let Pi^ and 
Fly denote the marginals of Pr on X and y, respectively. Let Vy = {Pry : Pr G V}. If 
E C X xy, then let P | ^ = {Pr | ^ : Pr G Pr(^) > 0}. Here Pr | E (denoted by some 
authors as Pr(- | -E")) is the distribution on X x y obtained by conditioning on E. 

Loss Functions: As in GH, we are interested in an agent who must choose some action from 
a set A, where the loss of the action depends only on the value of random variable Y. For ease 
of exposition, we assume in this paper that X, y, and A are always finite. We assume that with 
each action a E A and value y E y is associated some loss to the agent. (The losses can be 
negative, which amounts to a gain.) Let L : 3^ x ^ ^ iR be the loss function]! 

Such loss functions arise quite naturally. For example, in a medical setting, we can take y 
to consist of the possible diseases and X to consist of symptoms. The set A consists of possible 
courses of treatment that a doctor can choose. The doctor's loss function depends only on the 
patient's disease and the course of treatment, not on the symptoms. But, in general, the doctor's 
choice of treatment depends on the symptoms observed. 

Decision Rules: Suppose that the agent observes the value of a variable X that takes on val- 
ues in X. After having observed X, she must perform an act, the quality of which is judged 
according to loss function L. The agent must choose a decision rule that determines what she 
does as a function of her observations. We allow decision rules to be randomized. Thus, a 
decision rule is a function 5 : X ^ ^{-A) that chooses a distribution over actions based on 
the agent's observations. Let V{X,A) be the set of all decision rules. A special case is a 
deterministic decision rule, which assigns probability 1 to a particular action. If 6 is determin- 
istic, we sometimes abuse notation and write S{x) for the action that is assigned probability 
1 by the distribution 5{x). Given a decision rule S and a loss function L, let Ls be the ran- 
dom variable on X X y such that Ls{x,y) = Y.aeA^i^)i'^)^iy^ '^)- Here S{x){a) stands for 
the probability of performing action a according to the distribution S{x) over actions that is 
adopted when x is observed. Note that in the special case that 5 is a deterministic decision rule, 
Ls{x,y) = L{y,6{x)). 

A decision rule is a priori minimax optimal with respect to V and A if 

maxpreP-Epr[L5o] = min5gx)(A-,yi)maxpreP-Epr[L5]. (1) 

^We could equally well use utilities, which can be viewed as a positive measure of gain. Losses seem to be 
somewhat more standard in this literature. 
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That is, (5° is a priori minimax optimal if gives the best worst-case expected loss with respect 
to all the distributions in Pr. Note that we can write max here instead of sup because of our 
assumption that V is closed. This ensures that there is some Fi E V for which Ep^[Lso] takes 
on its maximum value. 

A decision rule 5^ is a posteriori minimax optimal with respect to V and A if, for all a; G A* 
such that Pr(X = x) > for some Pr G V, 

maXpi.ep|X=a;-Epr[I^5i] = ^2) 
min5gX)(A',^)niaXprg-p|x=a;-Epr[-^v5]. 

To get the a posteriori minimax-optimal decision rule we do the obvious thing: if x is observed, 
we simply condition each probability distribution Pr G P on X = x, and choose the action 
that gives the least expected loss (in the worst case) with respect io V \ X = x.Note that 
all distributions Pr mentioned in ^ satisfy Pr(X = x) = 1. Therefore, the minimum over 
5 G ^{X, A) does not depend on the values of 5{x') for x' ^ x; the minimum is effectively 
over randomized actions rather than decision rules. 

As the following example, taken from GH, shows, a priori minimax-optimal decision rules 
are in general different from a posteriori minimax-optimal decision rules. 

Example 2.1: Suppose that X = y = A = {0,1} and V = {Pr G A(A' x 3^) : Fiy{Y = 
1) = 2/3}. Thus, V consists of all distributions whose marginal on Y gives Y = 1 probability 
2/3. We can think of the actions in A as predictions of the value of Y. The loss function is if 
the right value is predicted and 1 otherwise; that is, L(i,j) = \i — j\. This is the so-called 0/1 
or classification loss. It is easy to see that the optimal a priori decision rule is to choose 1 no 
matter what is observed (which has expected loss 1/3). Intuitively, observing the value of X 
tells us nothing about the value of Y, so the best decision is the one which predicts according 
to the prior probability of F = 1. However, all probabilities on y = 1 are compatible with 
observing either X = or X = 1. That is, both (V \ X = 0)y and (V \ X = l)y consist of 
all distributions on y. Thus, the minimax optimal a posteriori decision rule randomizes (with 
equal probability) between Y = and Y = 1. 

Thus, if you make decisions according to the minimax rule, then before making an obser- 
vation, you will predict Y = 1. However, no matter what observation you make, after making 
the observation, you will randomize (with equal probability) between predicting Y = and 
Y = 1. Moreover, you know even before making the observation that your opinion as to the 
best decision rule will change in this way. | 

3 Two Game-Theoretic Interpretations of V 

What does it mean that an agent's uncertainty is characterized by a set V of probability distribu- 
tions? How should we understand P? We give V a game-theoretic interpretation here: namely. 
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an adversary gets to choose a distribution from the set But this does not completely specify 
the game. We must also specify when the adversary makes the choice. We consider two times 
that the adversary can choose: the first is before the agents observes the value of X, and the 
second is after. We formalize this as two different games, where we take the "adversary" to be 
a bookie. 

We call the first game the P-game. It is defined as follows: 

1 . The bookie chooses a distribution Pr G V. 

2. The value s of X is chosen (by nature) according to Pr;^ and observed by both bookie 
and agent. 

3. The agent chooses an action a ^ A. 

4. The value y of F is chosen according to Pr | X = x. 

5. The agent's loss is L{y, a); the bookie's loss is —L{y, a). 

This is a zero-sum game; the agent's loss is the bookie's gain. In this game, the agent's strategy 
is a decision rule, that is, a function that gives a distribution over actions for each observed 
value of X. The bookie's strategy is a distribution over distributions in V. 

We now consider a second interpretation of V, characterized by a different game that gives 
the bookie more power. Rather than choosing the distribution before observing the value of X, 
the bookie gets to choose the distribution after observing the value. We call this the V-X-game. 

1. The value x of X is chosen (by nature) in such a way that Pr(X = x) > for some 
Pr G V, and observed by both the bookie and the agent. 

2. The bookie chooses a distribution Pr G P such that Pr(X = x) > 00 

3. The agent chooses an action a E A. 

4. The value y of F is chosen according to Pr | X = x. 

5. The agent's loss is L{y, a); the bookie's loss is —L{y, a). 

Recall that a pair of strategies {Si, S2) is a Nash equilibrium if neither party can do better 
by unilaterally changing strategies. If, as in our case, (5*1, S2) is a Nash equilibrium in a zero- 
sum game, it is also known as a "saddle point"; 5*1 must be a minimax strategy, and S2 must be 
a maximin strategy HGrtinwald and Dawid 20041 . As the following results show, an agent must 
be using an a priori minimax-optimal decision rule in a Nash equilibrium of the V-game, and 

^This interpretation remains meaningful in several practical situations where there is no explicit adversary; see 
the final paragraph of this section. 

'^If we were to consider conditional probability measures, for which Pr{Y = y \ X = x) is defined even if 
Pr(X = x) = 0, then we could drop the restriction that x is chosen such that Pr(X = a;) > for some Pr G V. 
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an a posteriori minimax-optimal decision rule is a Nash equilibrium of the V-X-game. This 
can be viewed as a justification for using (a priori and a posteriori) minimax-optimal decision 
rules. 

Theorem 3.1: Fix X, y. A, L, and V C A(A' x y). 

(a) The V-game has a Nash equilibrium {it*, 5*), where tt* is a distribution over V with 
finite support. 

(b) If (tt*, 5*) is a Nash equilibrium of the V-game such that tt* has finite support, then 

(i) for every distribution Pr' e V in the support ofir*, we have 

Epr'[Ls*] — maxpre-p-BprfLj*]; 

(ii) if Pr* = I]prep.7r*(Pr)>o 7r*(Pr) Pr (i.e., Pr* is the convex combination of the distri- 
butions in the support ofn*, weighted by their probability according to n*), then 

Epr* [Ls*] 

= maxprep mmsev{x,A) Epr [Ls] 
— maxprgp Ep,. [Lg*]. 

Once nature has chosen a value for X in the 7^ -X -game, we can regard steps 2-5 of the 
V-X-game as a game between the bookie and the agent, where the bookie's strategy is char- 
acterized by a distribution inV \ X — x and the agent's is characterized by a distribution over 
actions. We call this the V-x-game. 

Theorem 3.2: Fix X, y. A, L,V C A{X x y). 

(a) The V-x-game has a Nash equilibrium (tt*, S*{x)), where n* is a distribution over V \ 
X — X with finite support. 

(b) If {tt* , S* (x)) is a Nash equilibrium of the V-x-game such that TT* has finite support, then 

(i) for all Pr' in the support o/tt*, we have 

Epr'[Ls*] — maxpr^-p\x=xEpr[Ls*]; 

(ii) if Fr* = EpreP,7r*(Pr)>o 7r*(Pr) Pr, then 

Epr* [Ls*] 

= mmsev{x,A)Epr*[Ls] 

— maXpr(,-p\x=x'^^^SGV(X,A)Epr[Ls] 

= mms(zi:,(x,A)'^8iXp^^'P\x=xEpr[Ls] 

— ia.axpre-p\x=xEpr[Ls*]. 
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Since all distributions Pr in the expression mms^T>{x,A)^^^PTep\x=xEpr[Ls] in part (b)(ii) are 
in V I X = X, as in Q, the minimum is effectively over randomized actions rather than 
decision rules. 

Theorems 13. II and 13.21 can be viewed as saying that there is no time inconsistency; rather, 
we must just be careful about what game is being played. If the P-game is being played, the 
right strategy is the a priori minimax-optimal strategy, both before and after the value of X 
is observed; similarly, if the P-X-game is being played, the right strategy is the a posteriori 
minimax-optimal strategy, both before and after the value of X is observed. Indeed, thinking 
in terms of the games explains the apparent time inconsistency. While it is true that the agent 
gains more information by observing X = x, in the V-X game, so does the bookie. This 
information may be of more use to the bookie than the agent, so, in this game, the agent can be 
worse off by being given the opportunity to learn the value of X. 

Of course, in most practical situations, agents (robots, statisticians,. . . ) are not really con- 
fronted with a bookie who tries to make them suffer. Rather, the agents may have no idea at all 
what distribution holds, except that it is in some set V. Because they have no idea at all, they 
decide to prepare themselves for the worst-case and play the minimax strategy. The fact that 
such a minimax strategy can be interpreted in terms of a Nash equilibrium of a game helps to 
understand differences between different forms of minimax (such as a priori and a posteriori 
minimax). From this point of view, it seems strange to have a bookie choose between different 
distributions in V according to some distribution vr*. However, if V is convex, we can replace 
the distribution tt* on P by a single distribution in V, which consists of the convex combina- 
tion of the distributions in the support of tt*; this is just the distribution Pr* of Theorems 13. II 
and 13. 21 Thus, Theorems 13.11 and [3 . 21 hold with the bookie restricted to a deterministic strategy. 

4 Characterizing A Priori Minimax-Optimal Decision Rules 

To get the a posteriori minimax-optimal decision rule we do the obvious thing: if x is observed, 
we simply condition each probability distribution Fi E V on X = x, and choose the action 
that gives the least expected loss (in the worst case) with respect toV\X = x. 

We might expect that the a priori minimax-optimal decision rule should do the same thing. 
That is, it should be the decision rule that says, if x is observed, then we choose the action that 
again gives the best result (in the worst case) with respect toV\X = x. However, as shown 
in GH, this intuition is incorrect in general. There are times, for example, that the best thing to 
do is to ignore the observed value of X, and just choose the action that gives the least expected 
loss (in the worst case) with respect to V, no matter what value X has. In this section we first 
give a sufficient condition for conditioning to be optimal, and then characterize when ignoring 
the observed value is optimal. 

Definition 4.1: Let {V) = {Pr G A{X x y) : Pr^t" e Vx and {Ft \ X = x) e {V \ X = x) 
for all X E X such that V \ X = x is nonempty}. | 
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Thus, {V) consists of all distributions Pr whose marginal on X is the marginal on X of some 
distribution in V and whose conditional on observing X = xh the conditional of some distri- 
bution in V, for all x & X. Clearly V C (P), but the converse is not necessarily true. When it 
is true, conditioning is optimal. 

Proposition 4.2: IfV = {V), then there exists an a priori minimax-optimal rule that is also a 
posteriori minimax optimal. If, for all Pr G P and all x & X, Vi^X = x) > 0, then every a 
priori minimax-optimal rule is also a posteriori minimax optimal. 



As we saw in Example l2.1l the minimax-optimal a priori decision rule is not always the same as 
the minimax-optimal a posteriori decision rule. In fact, the minimax-optimal a priori decision 
rule ignores the information observed. Formally, a rule S ignores information if S{x) = 5{x') 
for all x,x' E X. If 6 ignores information, define L'^ to be the random variable on y such that 
L'^{y) = Ls{x, y) for some choice of x. This is well defined, since Ls{x, y) = Ls{x\ y) for all 
X, x' G X. 

Theorem 4.3: Fix X, y, L, A, and "P C A(A' x 3^). If for all Viy G Vy, V contains a 
distribution Pr' such that X and Y are independent under Pr', and Vi'y = Piy, then there is 
an a priori minimax-optimal decision rule that ignores information. Under these conditions, 
if 5 is an a priori minimax-optimal decision rule that ignores information, then 5 essentially 
optimizes with respect to the marginal on Y; that is, maxp^gp Ep^[Ls] = maxpr^gp^ ^Pry [L's]. 

GH focused on the case that Vy is a singleton (i.e., the marginal probability on Y is the same 
for all distributions in V) and for all x, Vy (V \ X = x)y. It is immediate from Theorem l4.3l 
that ignoring information is a priori minimax optimal in this case. 

5 C -conditioning and Calibration 

Conditioning is the most common way of updating uncertainty. In this section, we examine 
updating by conditioning. The following definition makes precise the idea that a decision rule 
is based on conditioning. 

Definition 5.1: A probability update rule is a function 11 : 2^('^^^) x X ^ 2^('^^^) mapping a 
set V of distributions and an observation x to a set Il{V, x) of distributions; intuitively, Il{V, x) 
is the result of updating V with the observation x. I 

Definition 5.2: Let C = {Xi, . . . , Xk} be a partition of X; that is, Xi ^ ^ for i = 1, . . . ,k; 
Xi U . . . Xk = X; and Xi Ci Xj =0 for i ^ j. If x G X, let C{x) be the cell containing x; 
i.e., the unique element Xi E C such that x E Xi. The C-conditioning probability update rule 
is the function 11 defined by taking Il(V, x) = V \ X E C{x). A decision rule 6 is based on 
C-conditioning if it amounts to first updating the setVtoV\XEC{x), and then taking the 
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minimax-optimal distribution over actions relative to V \ X G C{x). Formally, 5 is based on 
C-conditioning if, for aWx E X with Pr(X = x) > for some Pr G V, 

max EpJLx] = min max EpJL/;]. 

Prg-p|xec(x) ^ ' 5eV{x,A)Prer\xeC{x) ^ ' 



All examples of a priori minimax decision rules that we have seen so far are based on C- 
conditioning: Standard conditioning is based on C-conditioning, where we take C to consist 
of all singletons; ignoring information is also based on C-conditioning, where C = {X}. This 
suggests that, perhaps, the a priori minimax decision rule must also be based on C-conditioning. 
The following well-known example shows that this conjecture is false. 

Example 5.3: [The Monty Hall Problem] IIMosteller 19651 Ivos Savant 1990ll : Suppose that 
you're on a game show and given a choice of three doors. Behind one is a car; behind the 
others are goats. You pick door 1. Before opening door 1, Monty Hall, the host (who knows 
what is behind each door) opens one of the other two doors, say, door 3, which has a goat. He 
then asks you if you still want to take what's behind door 1, or to take what's behind door 2 
instead. Should you switch? You may assume that initially, the car was equally likely to be 
behind each of the doors. 

We formalize this well-known problem as a P-game, as follows: ^ = {1, 2, 3} represents 
the door which the car is behind. X = {G2, G3}, where, for j E {2, 3}, Gj corresponds to 
the quizmaster showing that there is a goat behind door j. A = {1,2, 3}, where action a E A 
corresponds to the door you finally choose, after Monty has opened door 2 or 3. The loss 
function is once again the classification loss, L{i,j) = 1 if i 7^ j, that is, if you choose a door 
with a goat behind it, and L{i, j) = if i = j, that is, if you choose a door with a car. V is the 
set of all distributions Pr on A" x 3^ satisfying 

Pr3;(r = 1) = PlyiY = 2) = PTy{Y = 3) = i 

Ft{Y = 2 I X = G2) = 
Pr(r = 3 I X = Gg) = 0. 

It is well known, and easy to show, that the minimax-optimal strategy is always to switch doors, 
no matter whether Monty opens door 2 or door 3. Since the game is an instance of the V-game, 
this means that the decision rule 5* given by 

6*{G2)=3; S*{G,)=2 

is an a priori minimax rule. It is clear that S* is not based on C-conditioning: there exist only 
two partitions of X. The corresponding two update rules based on C-conditioning amount to, 
respectively, (a) ignoring X and choosing each door with probability 1/3, or (b) conditioning 
on X in the standard way and thus choosing each of the two remaining doors with probability 
1/2. Neither strategy (a) nor (b) is minimax optimal. Thus, the a priori minimax decision rule 
in the P-game is not always based on C-conditioning. | 
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While the example shows that C -conditioning is not always optimal in the minimax sense, 
it can be justified by other means; as we now show, C-conditioning is closely related to cali- 
bration. Indeed, a probability update rule is calibrated if and only if for each V, it amounts to 
C-conditioning for some partition C of X. Calibration is usually defined in terms of empirical 
data. To explain what it means, consider a weather forecaster, who predicts the probability 
of rain every day. How should we interpret the probabilities that she announces? The usual 
interpretation — which coincides with most people's intuitive understanding — is that, in the 
long run, on those days at which the weather forecaster predict probability p, it will rain ap- 
proximately 100p% of the time [ Dawid 1982il . Thus, for example, among all days for which 
she predicted 0.1, the fraction of days with rain was close to 0.1 (given the weather forecaster's 
precision, we should require it to be between, say, 0.05 and 0.15). A weather forecaster with 
this property is said to be calibrated. If a weather forecaster is calibrated, and you make bets 
which, based on her probabilistic predictions, seem favorable, then in the long run you cannot 
lose money. If a weather forecaster is not calibrated, there exist bets which seem favorable but 
which result in a loss. Note that calibration is a minimal requirement: a weather forecaster who 
predicts 0.3 for every single day of the year may be calibrated if it indeed rains on 30% of the 
days, but still not very informative. Thus, given two calibrated forecasters, we prefer the one 
that makes "sharper" predictions, in a sense to be defined below. 

In our case, we do not test probabilistic predictions with respect to empirical relative fre- 
quencies, but with respect to other sets of "potentially underlying" probability measures. We 



are not the first to do this; see, for example, dVovk, Gammerman, and Shafer 20051 . The defi 



nition of calibration extends naturally to this situation. To see how, we first define calibration 
with respect to a single underlying probability measure. Let V = {Pr} for a single distribution 
Pr and let 11 be a probability update rule (Definition 15. II) such that n({Pr}, x) contains just a 
single distribution for each x E X (for example, 11 could be ordinary conditioning). We define 

R= {7^:7^= (n(P,x) )yforsome s G A*}. (3) 

R is just the range of 11, restricted to distributions of Y, the random variable that we are 
interested in predicting; its elements are the distributions on Y that Pr is mapped to, upon 
observing different values of x. Note that R is defined relative to a probability update rule 11 
and a set V of distributions. By our assumptions on V and Il,Tl = {{Ri}, {R2}, . . .} is a set 
of singleton sets, each containing one distribution on y. For {R} E R, let be the set of 
X E X that map Pr to R, i.e. 

Xr = {xEX : {U{{PT},x))y = {R}}. 

Note that the sets {Xj^} partition X. U is calibrated relative to V if for all R with {R} E R, 

(Pr I X G XR)y = R. 

Thus, conditioned on the event that the agent predicts Y using distribution R, the distribution 
of Y must indeed be equal to R. 
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It is straightforward to generalize this notion to sets V of probability distributions that are 
not singletons, and update rules 11 that map to sets of probabilities. Definition ([3]) remains 
unchanged. For 7^ G R, we now take X-ji to be the set of x G A" that map V to IZ, that is, 

Xn = {xeX ■ {Il{V,x))y=n}. (4) 

Once again, the sets {X-ji} partition X. 

Definition 5.4: 11 is calibrated relative to V if for all Pr G V and 7^ G R, 

Pry(- \ x eXT^)en. 

n is calibrated if it is calibrated relative to all sets of distributions P C A(A:' x 3^). | 

Proposition 5.5: For all partitions C of X and all V, C-conditioning is calibrated relative to 
V. 

Calibration as defined here is a very weak notion. For example, the update rule 11 (P, x) = 
A{X X y) that maps each combination of x and V to the set of all distributions on X x y 
is calibrated under our definition. This update rule loses whatever information may have been 
contained in V, and is therefore not very useful. Intuitively, the fewer distributions that there 
are in V, the more information V contains. Thus, we restrict ourselves to sets V that are as 
small as possible, while still being calibrated. 

Definition 5.6: Update rule H' is wider than update rule 11 relative to V if, for all x G A", 

n(P,a;) C n'(P,x). 

n' is strictly wider relative to V if the inclusion is strict for some some x. 11 is (strictly) 
narrower than 11', relative to V if 11' is (strictly) wider than 11 relative to V. 11 is sharply 
calibrated relative to P if 11 is calibrated relative to V and there is no update rule H' that is 
calibrated and strictly narrower than 11 relative to P. 11 is sharply calibrated if 11 is sharply 
calibrated relative to all P C A(A:' x 3^). | 

We now want to prove that every sharply calibrated update rule must involve conditioning. 
To make this precise, we need the following definition. 

Definition 5.7: n is a generalized conditioning update rule if, for all P C A[X x y), there 
exists a partition C (that may depend on V) such that for all x E X, n(P, x) =V \ C{x).t 

Note that in a generalized conditioning rule, we condition on a partition of X, but the partition 
may depend on the set V. For example, for some V, the rule may ignore the value of x, whereas 
for other V, it may amount to ordinary conditioning. It easily follows from Proposition l4.2l that 
every generalized conditioning rule is calibrated. The next result shows that every sharply 
calibrated update rule must be a generalized conditioning rule. 
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Theorem 5.8: There exists an update rule that is sharply calibrated. Moreover, every sharply 
calibrated update rule is a generalized conditioning update rule. 

Theorem [52] says that an agent who wants to be sharply calibrated should update her prob- 
abilities using conditioning (although what she conditions on may depend on the set of proba- 
bilities that she considers possible). 

Given the game-theoretic interpretation of Section [3l we might wonder if there is a variant 
of the games considered earlier for which the equilibrium involves generalized conditioning. 
As we show in the full paper, there is (although the game is perhaps not as natural as the ones 
considered in Section [3]). Roughly speaking, we consider a three-player game, with a bookie 
and two agents. The bookie again chooses a probability distribution from a set "P; the bookie 
also chooses the loss function from some set. The first agent observes V and x and updates V 
to Vx- The second agent learns Vx and h (but not V and x) and makes the minimax-optimal 
decision. As we show, in Nash equilibrium, the first agent's updated set of probabilities, Vx, 
must be the result of C-conditioning, where, as in Theorem 15. 8 [ C may depend on V. 

6 Discussion 

We have examined how to update uncertainty represented by a set of probability distributions, 
where we motivate updating rules in terms of the minimax criterion. Our key innovation has 
been to show how different approaches can be understood in terms of a game between a bookie 
and an agent, where the bookie picks a distribution from the set and the agent chooses an action 
after making an observation. Different approaches to updating arise depending on whether the 
bookie's choice is made before or after the observation. We believe that this game-theoretic ap- 
proach should prove useful more generally in understanding different approaches to updating. 
We hope to explore this further in future work. 

We end this paper by giving an overview of the senses in which conditioning is optimal and 
the senses in which it is not, when uncertainty is represented by a set of distributions. We have 
established that conditioning the full set P on X = x is minimax optimal in the V-x-gamQ, but 
not in the P-game. The minimax-optimal decision rule in the P-game is often an instance of 
C-conditioning, a generalization of conditioning. The Monty Hall problem showed, however, 
that this is not always the case. On the other hand, if instead of the minimax criterion, we insist 
that update rules are calibrated, then C-conditioning is always the right thing to do after all. 

There are two more senses in which conditioning is the right thing to do. First, Walley 
[119911 shows that, in a sense, conditioning is the only updating rule that is coherent, according 
to his notion of coherence. He justifies coherence decision theoretically, but not by using the 
minimax criterion. Note that the minimax criterion puts a total order on decision rules. That 
is, we can say that 5 is at least as good as 5' if 

maxpi.g-pi?pi.[L5] < maxpi.gp-E'pr[i^5']. 
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By way of contrast, Walley II1991II puts a partial order on decision rules by taking 5 to be at 
least as good as 6' if 

ma.xp^^-pEp^[Ls — Ls'] < 0. 

Since both maxp^^-pEpT.[Ls — Ls'] and maxpi.gp-Epr[iv5' — Ls] may be positive, this is indeed 
a partial order. If we use this ordering to determine the optimal decision rule then, as Walley 
shows, conditioning is the only right thing to do. 

Second, in this paper, we interpreted "conditioning" as conditioning the full given set of 
distributions V. Then conditioning is not always an a priori minimax optimal strategy on the 
observation X = x. Alternatively, we could first somehow select a single Pr G V, condition 
Pr on the observed X = x, and then take the optimal action relative to Pr | X = s. It follows 
from Theorem 13. II that the minimax-optimal decision rule 5* in a V-game can be understood 
this way. It defines the optimal response to the distribution Pr* G A(X x 3^) defined in 
Theorem 13 . 1 r b)(ii) . If V is convex, then Pr* G V. In this sense, the minimax-optimal decision 
rule can always be viewed as an instance of "conditioning," but on a single special Pr* that 
depends on the loss function L rather than on the full set V. 

It is worth noting that Grove and Halpern ni998ll give an axiomatic characterization of 
conditioning sets of probabilities, based on axioms given by van Fraassen [ 19871 1198511 that 
characterizing conditioning in the case that uncertainty is characterized by a single probability 
measure. As Grove and Halpern point out, their axioms are not as compelling as those of van 
Fraassen. It would be interesting to know whether an axiomatization that is similar in spirit 
can be used to characterize the update notions that we have considered here. 
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A Proofs 

To prove Theorems 13.11 and Theorem 13.21 we need two preliminary observations. The first 
is a characterization of Nash equilibria. In the V-game, a Nash equilibrium or saddle point 
amounts to a pair (tt*, 5*) where n* is a distribution on V and 5* is a randomized decision rule 
such that 

E-,T*Epr [Ls*] 

= mms(zTi{A;,A)ETT'-Epj.[Ls] (5) 
= maxprg-pEpjL^*], 
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where -E'^*-E'pr is an abbreviation of Z]preP,7r*(Pr)>o 7r*(Pr) Pr. In the V-x-game, a Nash equi- 
librium is a pair (vr*, S*) where vr* is a distribution on V \ X = x and S* is a randomized 
decision rule, such that ^ holds with V replaced hy V \ X = x. 

The second observation we need is the following special case of Theorem 3.2 in HGriinwald and Dawid 2004 1 
itself an extension of Von Neumann's original minimax theorem. 

Theorem A.l: Ify' is a finite set, V is a closed and convex subset o/ A(3^'), A! a closed and 
convex subset of for some k G IN, and L' : y x A' ^ IR is a bounded function such that, 
for each y G y, L{y, a) is a continuous function of a, then there exists some Pr* G V and 
some p* G A' such that, 

Ep,*[L'iY',p*)] 
= miUp^A' Epr*[L'{Y',p)] (6) 
= msiXp,^r'Epr[L'{Y',p*)]. 

With these observations, we are ready to prove Theorem 13. II 
Theorem IXTt Fix X, y. A, L, andV C A{X x y). 

(a) The V-game has a Nash equilibrium (7r*,(5*), where it* is a distribution over V with 
finite support. 

(b) If (vr*, 6*) is a Nash equilibrium of the V-game such that tt* has finite support, then 

(i) for every distribution Pr' G V in the support of tt* , we have 

Epj.'[Ls*] = maxprep-Eprf/^^*]; 

(ii) ifPr* = Z]prep.7r*(Pr)>o 7r*(Pr) Pr (i.e., Pr* is the convex combination of the distri- 
butions in the support ofir*, weighted by their probability according to n*), then 

Epr* [Ls*] 
= mms(zD(x,A)Epr*[Ls] 
= maxprep mms^v(x,A) Epr [Ls] 
= mms(zv(x,A)^^^Pr€pEp,[Ls] 
= maxprep-EpjL^.]. 

Proof: To prove part (a), we introduce a new loss function L' that is essentially equivalent to 
L, but is designed so that Theorem lA.ll can be applied. Let y = X x y, let A' = V{X, A), 
and define the function L' : 3^' x ^ ^ as 

L\{,x,y),5) := Ls{x,y) = ^ 5{x){a)L{y, a). 

aeA 

Obviously L' is equivalent to L in the sense that for all Pr G A{X x y), for all S G ^^{X, A), 

Ep,[Ls] = Ep,[L'{{X,Y),6)]. 
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If we view A' = V{X,A) as a convex subset of iRl-^I C-^''^), then L'{{x,y), a) becomes a 
continuous function of a G A'. Let V be the convex closure of V. Since X x y is finite, V 
consists of all distributions Pr* on {X, y) of the form ci Pii H — ■ + Pifc for k = \X x y\, 
where Pri, . . . , Pr^ G V and ci, . . . , are nonnegative real coefficients such that ci+- ■ ■+Ck = 
1. Applying Theorem lA.il to L' and V, it follows that Q holds for some Pr* G V and some 
5* e A' = 'D{X,A) (that is, the p* in © is 5*). Thus, there must be some distribution n* 
on V with finite support such that Pr* = Z]pr6P,7r*(Pr)>o P^- It is easy to see that the 

two equalities in Q are literally the two equalities in ([5]). Thus, (tt*, 5*) is a Nash equilibrium. 
This proves part (a). 

To prove part (b)(i), suppose first that (tt*, 6*) is a Nash equilibrium of the P-game such 
that Tc* has finite support. Let V = maxp^gp EpjL^.]. By ([5]), we have that 

J2 7i*{Ft)Ep,[Ls*] = V. (7) 

Pre-p,7r*{Pr)>0 

Trivially, for each Pr' G V, we must have Epj.>[Ls*] < raaxp^^p Ep^[Ls*]. If this inequality 
were strict for some Pr' E V in the support of tt*, then I]preP,7r*(Pr)>o 7r*(Pr)£'pr[L5.] < V, 
contradicting (|7]). This proves part (b)(i). 

To prove part (b)(ii), note that straightforward arguments show that 

> mins(zv{x,A) T^^^Prep Ep, [Ls] 

> msiXpr(z-pmiias(zv{x,A)Epr[L5] 

> mins^v{x,A)Epr*[Ls]- 

(The second inequality follows because, for all Pr' G V, minsev{x,A)^^^Prev Ep^[Ls] > 
mins^Ti{x,A)Epr'[Ls]-) Since (tt*, 6*) is a Nash equilibrium, part (b)(ii) is immediate, using the 
equalities in (|5]). | 

Theorem |32l Fix X, y, A L,V <^ /\{X x y). 

(a) The V-x-game has a Nash equilibrium (vr*, 5*{x)), where vr* is a distribution over V \ 
X = X with finite support. 

(b) If {it*, S*{x)) is a Nash equilibrium of the V-x-game such that vr* has finite support, then 

(i) for all Pr' in the support ofn*, we have 

Epr'iLs*] = maxp^(='p\x=xEpr[Ls*]; 

(ii) if Fr* = Epre7',,r*(Pr)>o 7r*(Pr) Pr, then 

Epr* [Ls*] 
= minsev(x,A)Epr*[Ls] 
= maxpj.^'pix=x^^'^5€V{x,A)Epr[L5] 
= mins(zT:i{x,A)'^^^Prev\x=xEpr[Ls] 
= maxpj.(z'p\x=xEpr[Ls'-]. 
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Proof: To prove part (a), we apply Theorem lA.U setting L' = L, y' = y. A' = A(^), and 
V to the convex closure ofV\X = x. Thus, © holds for some p* E A', which we denote 
S*{x). As in the proof of Theorem 13. 1[ there must be some distribution n* onV \ X = x with 
finite support such that Pr* = Y.Prev\x=x,TT*{Pv)>o 7r*(Pr) Pr. The remainder of the argument 
is identical to that in Theorem l3.1[ 

The proof of part (b) is completely analogous to the proof of part (b) of Theorem |3.1[ and 
is thus omitted. I 

Proposition 14.21 : IfV = (V), then there exists an a priori minimax- optimal rule that is also a 
posteriori minimax optimal. If, for all Pt E V and all x E X, Pr(X = x) > 0, then every a 
priori minimax- optimal rule is also a posteriori minimax optimal. 

Proof: Let = {x E X : maxprg-p Pr(X = x) > 0}. Let nis be a random variable on X 
defined by taking ms{x) = if x ^ X^ , and ms{x) = maxpj.'g-p|x=x Ep^'[Ls] if x G X^. We 
first show that for every 5 E 'D{X,A), 

ma.xEpj.[Ls] = max Ptx{X = x)ms{x). (8) 

x&Pc 



Note that 

Epr [Ls] 

= Eix,y)GXxy Pr((X, Y) = (x, y))Ls{x, y) 

= E{x&X:Pr;,(x)>o} ^^^xiX = x) J^yev Pr(^ = x\X = x)Ls{x, y) 

= T.{xf^X:Prx{x)>0} ^^xiX = x)£'pr|x=z [-^i] 

< T,{x(^X:Pr:x{^)>0} ^^^iX = x) maXpi.'g-p[x=z Epj.'[Ls] 

= J2{xGX:Prx(x)>0} Pt^x{X = x)nis{x) 

= Y.x(^x PrA'(^ = x)ms{x). 



Taking the max over all Pr E V, we get that 

maxi?pi.[L5] < max 'S~] Pr;f(X = x)ms{x). 

xGX 



It remains to show the reverse inequality in ([8]). Since V is closed, there exists Pr* E V 
such that 

max Pr;t'(-^ = x)ms{x) = Pr^(X = x)ms{x). 

PrGl-^ — ^ — ^ 

xex xex 

Moreover, since P | X = x is closed, if x G X^, there exists Pr^ G P | X = x such that 
= Epr-[Ls]. Define Pr^ G A{X x y) by taking 

Prt((X,F) = {x,y)) = | pr^(x = x) Pr'^l" = 2/) if x G 
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Clearly Pr;^ = Pr;^ and (Pr^ \ X = x) = (Pr^ \ X = x) e V \ X = x if x e Thus, by 
definition, Pr^ G (V). Since, by assumption, {V) = V, it follows that Pr^ G V. In addition, it 
easily follows that 

maxprgp J2xex Pt^x{X = x)ms{x) 
= Prk^ = ^) T.yey Pr^(^ = y\X = x)Ls{x, y) 

= E^AU] 

< maxprep Ep,- [Ls] . 
This establishes ([8]). 

Now let 5* be an a priori minimax decision rule. Since the P-game has a Nash equilibrium 
(Theorem 13.11 ). such a 5* must exist. Let X' be the set of all x' E X for which 5* is not 
minimax optimal in the P-x'-game, i.e., x' G X' iff x G and ma,Xpj.fiz'p\x=x' -Spr'[-^<5*] > 
min5£X){A',yt) maxpr'g-p|x=a;' £^Pr'[-^^5]. Define 5' to be a decision rule that agrees with S* on 
A* \ Af' and is minimax optimal in the V \ X = x' game for all x' G X'; that is, 6'{x) = 5{x) 
for X ^ A"' and, for x G A"', 

S (x) G argmin5g^(;^. max Ep^' [L5] . 

By construction, nis' (x) < ms* (x) for all a; G A" and m^/ (x) < nis* (x) for all x G A"'. Thus, 
using dS]), we have 

maxprg-p EprlLsf] 
= maxprep T,xex Pi'(^ = x)ms' (x) 
< maxprep J2xex Pr(-^ = x)m5. (x) 
= maxpi.gp-Epr[I^5*]. 

Thus, S' is also an a priori minimax decision rule. But, by construction, S' is also an a posteriori 
minimax decision rule, and it follows that there exists at least one decision rule (namely, S') that 
is both a priori and a posteriori minimax optimal. Moreover, if Pr(X = x) > for all Pr G P 
and X E X and X' 7^ 0, then the inequality in ^ is strict. It follows that X' is empty in this 
case, for otherwise 5* would not be a priori minimax optimal, contradicting our assumptions. 
But, if X' is empty, then 5* must also be a posteriori minimax optimal. | 

Theorem HiS Fix X, y, L, A, and V C A{X x y). If, for all Piy E Vy, V contains a 
distribution Pr' such that X and Y are independent under Pr', and Pr^ = Pr3;, then there is 
an a priori minimax- optimal decision rule that ignores information. Under these conditions, 
if b is an a priori minimax- optimal decision rule that ignores information, then b essentially 
optimizes with respect to the marginal on Y; that is, maxp^ev Epj.[Ls] = maxpr^gpy -^Pr^; [L's]- 

Proof: Let V be the subset of V of distributions under which X and Y are independent. Let 
V{X, A)' be the subset of V{X, A) of rules that ignore information. Let 6* E V{X, A)' be 
defined as the optimal decision rule that ignores information relative to V, i.e. 

max Epj. \Ls* ] = min max Ep,- \Ls] . 



(9) 



19 



We have 

maxpi-g-p Epy-[Ls*] 

> mmsev{x,A) maxprgp Ep^ [Ls] 

> mms^x>{x,A)^^^Pr<^-P' Epr[Ls] (10) 
= min5g^(_;^.^^)/ maxprg-p' -Epr[-/^5] [see below] 

= maxprev Epr[Ls*]. 

To see that the equality between the third and fourth line in (flOl) holds, note that for Pr G V, 
we have 

Epr[Ls] 

= Y..^x Pr(X = x) Ey^y Pr(r = |/)(EaeA 5(a;)(a)L(y, a)) 

The decision rule that minimizes this expression is independent of x; it is the distribution 5* 
over actions that minimizes 

T.Pr{Y = y)iJ2S*ia)L{y,a)). 

yey a€A 

This calculation also shows that, since 6* ignores information, for Pr G V, we have that 

max Ep J Ls*]= max Ept.,,\LL] = max EpJLs*]- (H) 

This implies that the first and last line of (flOl) are equal to each other, and therefore also equal 
to the second line of (flOl) . It follows that S* is a priori minimax optimal. Since every a priori 
minimax optimal rule that ignores information must satisfy (fTT)) . the second result follows. I 

Proposition 15.51 : For all partitions C of X and all V, C -conditioning is calibrated relative to 
V. 

Proof: Let C = {Xi, . . . , X^} be a partition consisting of A; > 1 elements. Let TZj = {V \ 
X G Xj)y and let X-jz^ = Xj, for j = 1, . . . , A;; let R = {TZi, . . . , TZk}- Plugging this into 
Definition 15.41 we find that for C-conditioning to be calibrated, we must have that Pr(- | X G 
Xj) eV \ X e Xj for all V C A{X x 3^), all Pr G V, and all j G {1, . . . , k}. But this is true 
by definition ofV\XeXj.i 

We next want to prove Theorem 15.81 We need a preliminary lemma that shows that, in a 
sense, conditioning on some sets is always at least as good as any other update rule. 

Lemma A.2: For every probability update rule U and closed set V C A{X x y), if U 
is calibrated relative to V, then there exists a partition C of X such that C-conditioning is 
narrower than 11 relative to V. 

Proof: Suppose that n is calibrated relative to V. Suppose that R = {T^i, . . . , IZk] (where 
R is as defined in ([3])). Let C := {X-ji-^, . . . , Xti^}. Then C is a partition of X. We want to 
show that C-conditioning is narrower than 11 relative to V. Thus, we need to show that for 



20 



all X G '^■Rp V \ X E X-ji- C n(P, x). By assumption, Ii{V,x) = TZj. The result is now 
immediate from the definition of calibration. | 

Theorem 15.81 : There exists an update rule that is sharply calibrated. Moreover, every sharply 
calibrated update rule is a generalized conditioning update rule. 

Proof: To show that there exists an update rule that is sharply calibrated, we actually construct 
a generalized conditioning rule that is sharply calibrated. It suffices to show that for each closed 
P C A(A:' X 3^), there exists some partition C such that C-conditioning is sharply calibrated 
relative to V. We can place a partial order <p on partitions C by taking Ci <-p C2 if Ci- 
conditioning is narrower than C2 conditioning relative to V. Since X is finite, there are only 
finitely many possible partitions of of X. Thus, there must be some minimal elements of V. 
We claim that each minimal element of <p is sharply calibrated relative to V. For suppose that 
C is minimal relative to <p. If 11 is an update rule that is strictly narrower than C relative to V, 
then, by Lemma 1X21 there exists a partition C such that C is narrower than 11 relative to V. 
But then C <-p C, contradicting the minimality of C. This proves the desired result. 

To show that every sharply calibrated update rule is a generalized conditioning rule, sup- 
pose that n is sharply calibrated. Given V, by Lemma |A.2[ there must be some partition C 
such that C-conditioning is narrower that 11, relative to V. By Proposition [531 C-conditioning 
is calibrated relative to V. Since 11 is sharply calibrated, there can be no 11' that is strictly 
narrower than 11 relative to V and that is also calibrated relative to V. Thus, 11 must in fact 
coincide with C-conditioning relative to V. This proves that 11 is a generalized conditioning 
rule. I 
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